Robust PCA and Robust Subspace Tracking

Principal Components Analysis (PCA) is one of the most widely used dimension reduction techniques. Given a matrix of clean data, PCA is easily accomplished via singular value decomposition (SVD) on the data matrix. While PCA for relatively clean data is an easy and solved problem, it becomes much harder if the data is corrupted by even a few outliers. The reason is that SVD is sensitive to outliers. In today’s big data age, since data is often acquired using a large number of inexpensive sensors, outliers are becoming even more common. This harder problem of PCA for outlier corrupted data is called robust PCA. Often, for long data sequences, e.g., long surveillance videos, if one tries to use a single lower dimensional subspace to represent the data, the required subspace dimension may end up being quite large. For such data, a better model is to assume that it lies in a low-dimensional subspace that can change over time, albeit gradually. The problem of tracking a (slowly) changing subspace over time is often referred to as “subspace tracking” or “dynamic PCA”. The problem of tracking it in the presence of outliers can thus be called either “robust subspace tracking” or “dynamic robust PCA”. This article provides a comprehensive tutorial-style overview of the robust and dynamic robust PCA problems and solution approaches, with an emphasis on simple and provably correct approaches.